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ABSTRACT 


SIMRAND: SIMu lati on of Research AN d Development Projects is a method- 

ology developed at the Jet Propulsion Laboratory of the California Institute 
of Technology to aid the engineering and management decision process in the 
selection of the optimal set of systems or tasks to be funded on a research 
and development (R&D) Project. An R&D project may have a set of systems or 
tasks under consideration for which the total cost exceeds the allocated 
budget. Other factors such as personnel and facilities may also enter as 
constraints. Thus the project's management must select, from among the 
complete set of systems or tasks under consideration, a partial set that 
satisfies all project constraints. The SIMRAND methodology uses analytical 
techniques of probability theory, decision analysis of management sci ce, and 
computer simulation, in the selection of this optimal partial set. 

The SIMRAND methodology is truly a management tool. It initially 
specifies the information that must be generated by the engineers— thus 
providing information for the management direction of the engineers — and it 
ranks the alternatives according to the preferences of the decision makers. 

The decision makers could be either the project's management, the funding 
agency, or the end users. 
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INTRODUCTION 


A commonly occurring engineering and management decision on research and 
development (R&D) projects, whether commercial or military, is that of the 
optimal allocation of R&D funds, given budgetary constraints in funding alter- 
native systems for achieving the project's goals. Because of the budget 
constraints, not all of the proposed systems can be funded for R&D work. The 
engineering and management decision then is, "What set of the proposed systems 
should be funded?" SIMRAND: SIMulation of Research AN d Development Projects 

is a methodology developed at the Jet Propulsion Laboratory of the California 
Institute of Technology to aid engineers and management in the selection of the 
optimal set of systems to be funded. 

An R&D project must satisfy the following criteria for the SIMRAND 
methodology to be appropriate: 

(1) The goals of the project must be attainable by a system comprising 
hardware and manpower elements for which estimates of variables 
such as cost and performance can be made. Examples of such a 
system might be a logistics system or a system for producing solar- 
cell modules. 

(2) The systems for achieving the project goals must have a common 
measure of preference. Such a measure might be cost, with a pref- 
erence for minimizing cost, or it might be performance, with a 
preference for maximizing performance. 

(3) It must be possible to relate the variables that describe a system 
to the measure of preference. 

Although the following criteria are not mandatory, they do permit the 
full use of the SIMRAND methodology: 

(1) More than one system should be under consideration to satisfy the 
project goals. The systems may be fundamentally different, such 
as windmills versus solar cell nodules for generating electricity, 
or they may be parametric variations on a system design, such as 
the selection of the number and type of engines for a cargo 
aircraft. SIMRAND can select the optimal set of systems for R&D 
funding. 

(2) Uncertainty should be present with respect to the variables that 
describe the systems. The raison d'etre of R&D funding is to 
remove or at least reduce uncertainty, and in doing so to identify 
the "best" system or set of systems — otherwise, the project should 
enter directly into the implementation phase. SIMRAND incorpo- 
rates this uncertainty by treating the system variables probabil- 
istically. 

(3) The decision makers (who could be either project management, the 
funding agency, or the end users) should have a preference for the 
degree of risk they are willing to assume in terras of the extent 

to which the selected systems ultimately satisfy the project goals. 
SIMRAND represents this risk preference though the formal incorpo- 
ration of a risk analysis. 
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Given that these criteria are met, SIMRAND can determine the optimal set 
of systems for R&D funding. The inclusion of a specific system is tantamount 
to the funding of certain R&D tasks. SIMRAND will identify which R&D tasks 
are to be funded, and can also specify the level of funding required for each 
task. 


EXAMPLE NO. 1 

Consider a military R&D decision to develop a logistics system, where 
the measure of preference is to minimize the life-cycle cost per ton-mile of 
equipment moved. Assume that two systems are in contention for R&D funding — 
System A and System B — but that the funding level will only permit one of the 
two systems to be developed. Assume that System A is an upgraded version of 
an existing system, nd therefore the preference measure is known with virtual 
certainty — a point stimate will suffice. Assume that System B i s an advanced 
system, for which the preference measure can only be stated in probabilistic 
terms, e.g., a 50/50 chance that the cost per ton-mile for System B is less 
than that of System A. With no other information, and no knowledge of the risk 
preference of the decision makers, it is not possible to state whether System A 
or System B is the best system to develop. 

The SIMRAND methodology can be applied to this decision. SIMRAND not 
only structures the problem by placing it in a decision-making context, but 
also specifies the required information, and processes that information so 
that the question "Which system is preferred? "can be answered. 

The SIMRAND modeling of this decision is shown in Figure 1. There is a 
System Model, which is capable of analytically describing the two systems, and 
a Value Model , which takes the output of the System Model and determines which 
system is preferred. The two decision alternatives — which are to select either 
System A or System B — are shown as an arrow pointed into the System Model . 
Uncertainty as to the characteristics of System B is shown as another arrow, 
also pointed into the System Model. The outputs of the System Model are called 
Outcomes, and represent a characterization of the two systems as they would 
actually be realized — a characterization that can only be described probabil- 
istically for System B. The two outcomes form the input tc the Value Model . 

The Value Model incorporates both the preference measure of minimizing the cost 
per ton-mile and also the risk preference of the decision makers. The output 
of the Value Model is a preference ranking of the two alternatives. 


UNCERTAINTY 



Figure 1. A Block Diagram of the SIMRAND Methodology 
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The information required by SIMRAND from the engineers for this log- 
istics problem is straightforward, although the engineering effort required 
to develop the information may not be, depending upon the complexity of the 
system and the uncertainties involved. A point estimate of the cost per ton- 
mile is required for System A, and a probability distribution — formally called 
a cumulative distribution function — is required for System B. The probability 
distribution for System B is obtained by asking the engineers to make estimates 
in response to the f'i'.lowing kinds of questions: "For what cost per ton-mile 

for System B does the actual cost h.ave an x% chance of being less than or equal 
to it?" where x% would vary over the range from none to 100%. The cost per 
ton-mile for which System B would have a 50/50 chance of being less than or 
equal to it would be the 50% value. Other values of x% (1%, 10%, 25%, 75%, 90% 
and 99%) would provide enough information in most cases to draw a line through 
the points, thus forming the probability distribution of cost per ton-mile for 
System B. Figure 2 shows typical probability distribution for both System A 
(a point estimate) and System B, measured in units of the cost per ton-mile 
for System A. 
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Figure 2. Probability Distributions for the Estimated Cost per Ton-Mile 
for Example No. 1 
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A risk analys}'r: is incoip rated in SIMRAND by assessing the preferences 
of the decision luakei^ for different values of cost per ton-mile. The 
preferences are measured in s> ch a way that both strength of preference and 
risk preference are a<j::^esseii . Vhese preferences are assessed by asking the 
decision makers such qut-o^’ons as. "Would you rather have a system cost of 1 
unit per ton-mile or a 50/50 gamble- yielding a system cost of 0.5 units per 
ton-mile witl- probability 1/2 or a eystem cost of 2.0 units per ton-mile with 
probability 1/2?" From questions such as this one, a preference curve 
(formally called a utility function) can be constructed for each decision 
maker, such as that in Figure 3. Then an expected utility value can be 
determined for each system by integrating the utility function times the 
associated probability (as determined from the probability distribution) over 
the range of values :or cost per ton-mile. This integration determines an 
expected utility value fur each system f r each decision maker. Since greater 
utility values imply greater preference ranking, a preference ranking is 
established for each decision mai.er. 

If thrs risk ana.'y--i.® is ;'pplied to Figures 2 and 3 for Example 1, then 
System A receives an expe" 1 utility value of 0.50 and System B receives an 
expected utility va * “ 0 ‘Z. Therefore, for this hypothetical decision 

maker, the ai*^ r- ativu funding System A would be preferred to funding 
System B. ' i a st''^tlar nannor, preference rankings could be determined for 
other dec. cion makers, oroap decision rules could be applied to the preference 
rankinn;. ror > .-ch of the decision makers to identify a group consensus, if one 
exists. Typically, one person has the responsibility and authority to make 
the system selection, but that one person may be interested in the preferences 
of others before making the system selection. It is in this context that 
"decision makers" is used in the plural form. 

If the probability distributions for the cost p.ir ton-mile of Systems A 
and B are sufficiently well separated and do not overlap, then the risk 
analysis portion of the SIMRAND methodology is not required. For exai pie, if 
in Figure 2 the probabilitity distribution for System A were to iie entireley 
to the left of the probability distribution for System B, then System A (with 
a lower :ost oer ton-mile) would clearly be the preferred system. 



SYSTEM COST PER TON-MILE 

Figure 3. Typical Utility Function for a Decision Maker for Example No. 1 
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Even in this simple application of the SIMRAND methodology, several 
aspects of the methodology were used. A measure of preference for comparing 
the two systems was used — the cost per ton-mile of equipment moved. Probabil- 
ity theory was required to model the uncertainty associated with System B. 
Finally, risk analysis was used to measure the risk preference of the decision 
maker. The SIMRAND analysis of this example showed that the decision maker 
preferred System A. The undesirable prospect that System B might yield cost 
numbers significantly higher than System A outweighed ::he desirable prospect 
that System B might yield a cost number as low as one-half that of System A. 
For another decision maker, with a different utility function, the preference 
of System A over System B might be reversed. 


EXAMPLE NO. 2 

Example No. 2 is more complicated than Example No. 1, and demonstrates 
the full capability of the SIMRAND methodology. It is a simplified example of 
its application that has been made at the Jet Propulsion Laboratory for the 
Flat-Plate Solar Array Project. The objectives of the Project at the time the 
analysis was done were to minimize the production price of solar-cell modules, 
and to demonstrate their ability to perform reliably in operational environ- 
ments. Attainment of these objectives was sought by funding R&D tasks that 
could result in different ways of producing inexpensive solar-cell modules. 

Figure 4 is a simplified task network for the production of solar-cell 
modules. The production process is divided into five steps. Step 1 is the 
silicon purification step. Step 2 produces crystalline silicon, necessary for 



STEP 1 ^ STEP 2 ^ STEP 3 ^ STEP 4 ^ STEP 5 ^ 

SILICON CRYSTALLIZATION SAWING CELL MODULE 

PURIFICATION 

Figure 4. Task Network for Solar-Cell Module Production for Example No. 2 
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the photovoltaic transformation of sunlight into electricity. Step 3 is a 
sawing step that may or may not be necessary, depending upon whether the 
silicon is cast as ingots (for which sawing into sheets is necessary) or is 
pulled as ribbons (for which the sawing is not necessary). Step 4 takes the 
silicon in sheet or ribbon form and performs the necessary cleaning, doping, 
and conductor deposition to form a cell. Step 5 connects the cells and 
encapsulates them in a frame to form modules of cells. Price equations must 
be supplied for each step. The price equations will include known constants 
such as the density of silicon and the intensity of sunlight. The price 
equations will also include variables that can only be expressed probabilis- 
tically, such as solar cell efficiency and the prices of materials per unit 
quantity and the conts of processes. The price of the solar cell modules will 
be the sum of the value-added prices of each of the five steps. 

Figure 4 shows two different tasks ( lA and IB) by which the silicon can 
be purified in Step 1. Either Task lA or Task IB can purify silicon. The 
reasons for funding the two tasks in parallel are that technology and economic 
uncertainties make it impossible to know which task will purify the silicon at 
the least price, and that one of the tasks may involve an advanced technolcigy 
that cannot be guaranteed to work. Step 2 shows four tasks by which the 
silicon can be crystallized. Two of the tasks (2A and 2B) produce ingot 
silicon which must be sawed in Step 3, and two tasks ( 2C and 2D) produce 
ribbon silicon which requires no action in Step 3. Step 3 shows three tasks 
(3A, 3B, and 3C) for sawing the ingot silicon. Step 4 shows two tasks for 
producing the cells and Step 5 shows two tasks for producing the modules. 

There are 64 paths through the task network of Figure 4. Of these 64 
paths, 48 involve ingot tasks and 16 involve ribbon tasks. Several different 
questions may be asked of thi task network. If all of those tasks are funded, 
then what is the price of the solar cell modules — or more correctly, what is 
the probability distribution of the price of solar cell modules? If the R&D 
cost of funding all of these tasks exceeds the Project funding level, then 
what set of these tasks consistent with the funding level will result in the 
minimum price for the solar cell modules — or, more correctly, what set of 
these tasks consistent with the funding level will result in the most preferred 
probability distribution? If the tasks can be funded at different levels, then 
what distribution of funding over the task network will result in the most 
preferred probability distribution? The SIMRAND methodology can be applied to 
any of these questions, given that the necessary information can be obtained. 

The SIMRAND methodology proceeds in two phases, a reduction phase and a 
simulation phase. In the reduction phase, analytical techniques from proba- 
bility theory and simulation techniques are applied to reduce the complexity 
of the task network. In this simple example, there are only 64 paths through 
the task network; a more complicated case could have hundreds, or even thou- 
sands, of paths through the network. For example, a network consisting of 10 
steps and four parallel tasks for each step would have 1,048,576 paths. 

Clearly, any techniques that can be applied to reduce the number of paths can 
be extremely useful. In Figure 5, parallel tasks have been combined where 
possible to reduce the number of paths through the task network from 64 to two. 

The simulation phase provides the capability for the probabilistic 
analysis of task networks not amenable to the analytical techniques of proba- 
bility theory. Figure 6 expands the reduced task network of Figure 5 to form 
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STEP 1 I STEP 2 I STEPS . STEP 4 , STEPS 

SILICON CRYSTALLIZATION SAWING CELL ' MODULE 

PURIFICATION 

Figure 5. Reduced Task Network ior Solar-Cell Module Production 
of Example No. 2 
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Figure 6. Simulation Task Network for Solar-Cell Module Production 
of Example No. 2 


the simulation task network, which explicitly displays the two paths through 
the task network. Techniques from si\nulation theory are now applied to this 
simulation task network. The task network is .imulated in a computer program. 
The computer program performs the simulation by carrying out a series of Monte 
Carlo trials, consisting of (1) assigning a different random number to each of 
the probabilistic variables that appear in the price equations for the tasks, 
(2) calculating prices for each task using the task price equations, (3) 
suimning the task prices for each of the two paths through the simulation task 
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network, (4) selecting the pet'n >.<ith the mininun total price, (3) adding this 
nininun total price to a total price histogram, and the asKOciated step prices 
to step price histograms, and (6) incrementing the count of the number of 
times that the path for the minlmw total price has been selected. The random 
numbers are selected for eech of the probabilistic variables according to 
their frequency of uccurre.'.ce as determined by their associated probability 
distribution. 

These Monte Carlo trials are repeated many times with different sets of 
random numbers until histograms with sufficient precision have been constructed 
for the total price, the step prices, and the selected paths. Probability 
distributions and relevant statistics are calculated during the Monte Carlo 
trials or from the histograms. Probability distributions are calculated for 
the total price and for each of the step prices. Statistics such as mean 
price, variance in price, and percentiles are calculated for the total price 
and for each of the step prices. The number and percentage of times that each 
path of the network is selected as the minimum total price path are also 
calculated. 

The computer program performs a risk analysis by combining the utility 
functions of the decision makers with the total price histogram, and by 
deriving certainty equivalents for the total price probability distributions 
for each decision maker and for each task network under consideration. A 
certainty equivalent for a task network is the price at which a decision maker 
would have no preference between (a) receiving the price with certainty and (b) 
th . uncertainty of the probability distribution of total price associated with 
the task network. The importance of the certainty equivalent is that in the 
presence of uncertainty, it is the correct quantity to uce in ranking the task 
networks according to the risk preference of the decision laaker. Of the alter- 
native task networks under consideration (and that meet all the necessary pro- 
ject constraints, such as funding level), the task network with the lowest 
certainty equivalent is most preferred by the decision maker. 

As in Example No. 1, if the probability distributions of the total 
prices for the task networks do not overlap, then the risk analysis portion of 
the SIMRAND methodology is not required. An examination of tr.e probability 
distributions, such as in Example No. 1, or a statistic such as the mean total 
price, can be used to determine the preference ranking of the task networks. 

It is through this two-phased process that the SIMRAND Mthodology is 
able to efficiently model and analyze complex R&D funding decisions as multi- 
step, multitask networks. The reduction phase reduces the task network 
complexity, and in doing so reduces the simulation effort required in the sim- 
ulation phase. The simulation phase provides the capability of probabilistic 
analysis of task networks not aMr^ble to the analytical techniques of proba- 
bility theory. 


SUMMARY 

Certain criteria must be met tor the SIMRAND methodology to be appropri- 
ate for modeling and analyzing the optimal allocation of funds for an R&D pro- 
ject. These criteria include definitions of the R&D systems from which cost 
and performance estimates can be made. It must be possible to relate these 
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cost and performance estimates to a common measure of preference. The full 
capability of SIMRAND can be used when, in addition, more than one system is 
under consideration, when uncertainty exists with respect to the variables that 
define the systems, and when the risk preferences of the decision makers can 
be assessed. The decision makers may be either project management, the funding 
agency, or the er.d users. 

The SIMRAND methodology can make probabilistic estimates of the measure 
of preference of a system. The SIMRAND methodology can be applied to complex 
R&D task networks, and can provide information relevant to such questions as 
"Which set of R&D tasks should be funded?" and "What is the optimal distribu- 
tion of funding across a set of R&D tasks?" 

Two examples were used tc illustrate how the SIMRAND methodology could 
be employed. The examples illustrated the use of several systems analysis 
concepts, including the modeling of systems in the presence of uncertainty, 
risk analysis for incorporating the risk preference of decision makers into 
the system rankings, and the use of simulation techniques for analyzing task 
networks too complex for a straightforward application of probability theory. 

An important point to be made is that the preference rankings of the 
alternative systems generated by the SIMRAND methodology are those of the 
decision makers, not those of either the SIMRAND analysts or the engineers 
that perform the tasks and make the engineering and economic estimates for the 
systems. Thus the SIMRAND methodology is truly a management tool. The 
SIMRAND methodology initially specifies the information that must be generated 
by the engineers — thus providing information for their management direction — 
and it ranks the alternatives according to the preferences of the decision 
makers . 
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